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Background 

With the increased population of English learners (ELs), educators need programs that 
help students access academic content while learning to understand, speak, read, and write 
English (Tharp, Estrada, Dalteen, & Yamaguchi 2000; Echevarria, Short & Powers, 2006). ELs 
come into U.S. schools needing to learn the same grade-level content as their nonEL peers, but 
with the additional challenge of learning English — usually the language of instruction — as well. 
Although some approaches separate the two tasks, many researchers and practitioners call for 
integrating them, so that students do not miss content area instruction while they are learning 
English. One way to combine the two is sheltered instruction, which provides intentional 
linguistic and other supports to ELs to facilitate their learning of grade-level content (Echevarria, 
Short & Powers, 2008). As a recent review of the research makes clear, however, there is limited 
evidence to show the effectiveness of sheltered instruction (Goldenberg, 2013). 

Since the early 1990s, Project GLAD (Guided Language Acquisition Design) has made 
claim to be a program that helps teachers meet diverse language and content needs within the 
mainstream classroom (Brechtel, 2001). Project GLAD is a multi-component K-12 instructional 
model designed to build academic English and grade-level content knowledge for students at 
varying levels of English language proficiency. It is billed as an approach that benefits all 
students, but particularly ELs. Although before this study Project GLAD had not undergone a 
rigorous evaluation, there is research behind many of the 35 individual instructional strategies 
that make up the Project GLAD approach, which fall into four broad components (Table 1). 

Component 1. Five motivation strategies accomplish two goals: 1) they set expectations 
for behavior and engagement (Arnold, McWilliams, & Arnold, 1998; Bohn, Roehrig & Pressley, 
2004; Emmer & Stough, 2001), and 2) they build student interest by connecting to background 
knowledge, which may be particularly crucial for ELs (August & Shanahan, 2006; Kamil, 2003). 

Component 2. Six input strategies provide students with new information in multiple 
formats (images, graphic organizers, etc.) so that they can understand grade-level content, 
regardless of their level of language development (Behr, Lesh, Post, & Silver, 1983; Lee et 
al., 2005; Lee & Fradd, 1998; Sowell, 1989; Wenglinsky, 2000). 

Component 3. Ten guided oral practice strategies are designed to scaffold students’ use 
of key vocabulary and language structures in an accepting environment, often through 
cooperative learning (Kagen, 1993; Gersten et al., 2007). ELs require additional oral English 
language development beyond what is provided in standard language arts curricula (August & 
Shanahan, 2006; Gersten & Baker, 2000), and because oral language proficiency is correlated 
with the reading and writing ability of ELs (August & Shanahan, 2006; Genesee et al., 2006). 

Component 4. There are 14 reading and writing strategies to promote instruction in 
academic language through the deliberate scaffolding of more complex language skills, which 
can be crucial not only for ELs, but for all students (Davis & Miyake, 2004; Kuhn et al, 2006; 
Walqui & Van Lier, 2010). Here, as across all four component areas, the strategies intentionally 
build in opportunities for differentiated instruction, so teachers can meet the needs of students at 
different levels of English proficiency. 

Despite the research supporting particular instructional strategies, there has been no 
rigorous evidence to date about the impact of these strategies when they are integrated into a 
package called Project GLAD. This study addresses the need for such evidence. 
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Research Question 

• What is the impact of Project GLAD teacher training on fifth-grade students’ reading 
comprehension, vocabulary, writing and science achievement in the treatment classrooms 
during the first of implementation, compared to a “business as usual” control group? 

• Is the program impact different for ELs? 

Setting 

We conducted this two-year cluster randomized trial in fifth-grade classrooms from 30 
Idaho schools across 21 different districts. Almost half (47%) of our sample schools were 
classified as rural. The other half were located within towns (23%), cities (17%) or suburban 
locales (13%). School enrollment varied from 277 to 717, with a mean of 475 students. Table 2 
compares school-level characteristics of treatment and control schools (insert Table 2 here). 

Participants 

We began Year 1 (2011-2012) with 101 fifth-grade teachers across the 30 schools. Most 
teachers were white, female, and experienced, with an average of over 1 1 years of experience 
teaching (17 years for teachers in the control group; see Table 3). 

Our analytic sample consisted of 2,253 students. Ten percent were eligible for special 
education and 65 percent were eligible for free/reduced-price lunch (Table 4). Sixty-two percent 
of students were white and another third (33%) were Hispanic. Overall, 13 percent of students 
were current ELs or former ELs who had been reclassified within the prior two years (we 
combined the two groups because of small sample size and because of the way they were coded 
in state data). In order to be eligible to participate in the study, schools needed to have at least 
five ELs in grade five — and did, in the year of recruitment. However, in Year 1, one school had 
no ELs in fifth grade. The other 29 schools had between 4 and 30 percent current or recently 
reclassified ELs. While data on primary language was not available for the students in our study, 
93 percent of our EL sample was Hispanic and Idaho’s EL population is made up primarily of 
Spanish speakers (Idaho State Board of Education, 2012). 

Program 

The Project GLAD instructional model consists of 35 well-defined instructional 
strategies, organized, as noted above, into four broad categories. To leam to use the Project 
GLAD approach, teachers participate in a highly structured seven-day training sequence with 
follow-up coaching. There is no standardized amount of coaching; instead, the amount depends 
on the preferences and available funding of each district. For this study, teachers received three 
days of coaching spread over the school year, which represented an amount of coaching that 
many districts might realistically purchase. 

The seven days of professional development begin with a two-day workshop, which 
provides an overview of the approach and an introduction to the instructional strategies in 
August. A month or two later, teachers attend a five-day demonstration. In Project GLAD 
demonstrations, one trainer takes over the classroom of a participating teacher every morning for 
a week and teaches a complete compressed and intensive unit using the Project GLAD 
instructional strategies. Other teachers sit in the back of the classroom and watch, while a second 
trainer quietly explains what the trainer is demonstrating and why. In the afternoons, teachers 
work to plan a unit of their own and address questions and concerns with the trainers. All 42 
teachers in the treatment group attended all seven days of the training. 
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While the Project GLAD professional development provided for this study was in most 
ways typical of the training that teachers receive when schools or district purchase it, it was 
different in one important way: it was only delivered to fifth-grade teachers. Typically, all 
teachers in a school would be trained at the same time, which might provide greater 
administrative and collegial support for implementation. 

All teachers in the treatment group utilized at least some Project GLAD strategies, 
although the frequency and quality varied substantially. Teachers in the control schools delivered 
instruction “as usual”; in only five percent of control classrooms did we find any evidence of a 
Project GLAD (or GLAD-like) instructional strategy in use (Deussen & Nelsestuen, 2013). 

Research Design 

For our cluster randomized design, we recruited 30 schools with 101 fifth-grade teachers, 
and then randomly assigned 15 schools each to treatment and control conditions. Most teachers 
at every school participated, although a small number declined. Teachers in treatment and 
control schools were similar on most characteristics (Table 2), although teachers in the control 
schools had more experience (17.2 years compared to 11.6). 

Attrition. In Year 1, we lost no schools but 2 teachers left their classrooms for health 
reasons (one control and one treatment). In Year 1, we pretested 2778 students. The majority of 
these students also completed the battery of spring posttest assessments (83.1% overall; 83.6% 
for students in the control condition; and 82.5% for students in the treatment condition). Seven 
percent of students (6.5% overall; 5.7% for students in the control condition; and 7.3% for the 
treatment condition) did not complete any posttest assessments and were lost to attrition. 

Data Collection and Analysis 

Measures. Outcomes include three measures of literacy and two measures of fifth-grade 
science content (insert Table 5). For the Gates-MacGinitie (GM) reading and vocabulary 
measures, we had pretests that were different versions of the same test. For writing and science, 
we used the fall GM comprehension assessment as a covariate in place of a pretest. 

Data collection procedures. For each measure, we collected data from all 42 treatment 
and 50 control classrooms in Year 1. We provided the assessments and detailed instructions, and 
teachers administered the assessments during a designated testing window. 

Analysis. We examined treatment effects for reading comprehension, vocabulary, six 
distinct writing traits, and science separately using a two-level hierarchical linear model run with 
HLM6 (Raudenbush, Bryk, & Congdon, 2008). Treatment was included as a dichotomous 
variable at the school level, the level of assignment, using the formula below. We ran the model 
to test treatment effects on outcomes for all students, and separately for ELs. 

Level 1 [Student] 

Posttesti, = b 0 j + bijPretestij + e^ 

Level 2 [School] 

boj = goo + go i Treatment) + u 0j 

bij = gio 

Results 

Baseline equivalence of treatment and control. Table 6 displays the mean pretest scores 
for the GM reading comprehension and vocabulary scores. The treatment group started out with 
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a higher mean score in both reading comprehension and vocabulary (effect size, expressed as 
Hedge’s g = 0.117). We included the GM pretests in our model. 

Year 1 impact on all students. Tables 7 through 9 present findings from Year 1 of the 
randomized controlled trial for all students, EL and non-EL combined. With only small effect 
sizes and no statistically significant treatment effect, we found no evidence that Project GLAD 
improved students’ reading comprehension and vocabulary in the first year of implementation. 

We examined the impact of Project GLAD on students’ writing by looking at the six 
traits of effective writing. There were no statistically significant treatment effects for any of the 
six traits (Table 8). However, the result for the trait of “ideas” was marginally significant and 
was the largest effect size of any of the literacy measures (b=0.14, p=.062 and Hedges g= 0.22). 

Linally, on the science measures, we did not find any statistically significant impact in 
terms of treatment effect (Table 9), although the effect size for the end-of-unit science test was 
the largest of any measure (b=0.50, p=0.101, Hedges g = 0.24). 

Year 1 impact on ELs only. Tables 10 through 12 present findings from Year 1 for the 
EL subgroup only. Both the GM reading comprehension and vocabulary measures had marginal 
positive effects (b=6.87, p=0.099. Hedges g = 0.24 for comprehension and b=5.72, p=0.092, 
Hedges g = 0.21 for vocabulary). Of the six writing traits, there was also a marginally 
significant impact for the trait of “ideas” (b=0.21, p=0.053, Hedges g = 0.32) and “organization” 
(b=0. 15, p= 0.086, Hedges g = 0.27). There was no significant impact for the other writing traits 
nor for the two science measures. 

Conclusions 

The promise of sheltered instruction is that ELs’ instructional needs can be met by a 
mainstream classroom teacher. To realize this promise, many districts have invested heavily in 
professional development for their teachers in various models of sheltered instruction, including 
Project GLAD. But to date this investment has occurred with only limited evidence of modest 
impact (Goldenberg, 2013; Echevarria, Short & Powers, 2006). 

Our study, like others of similar programs, found no significant treatment effect for the 
general student population in the first year of implementation. Whether or not additional time to 
fully implement the program results in a significant effect is something that can be partly 
answered by our forthcoming Year 2 results. However, we did find marginally significant results 
for ELs in reading comprehension, vocabulary, and the writing traits of “ideas” and 
“organization.” The size of the effect for reading comprehension is equivalent to about 58 
percent of the growth in reading that fifth-grade students can be expected to make, on average, 
over the year (Bloom, Hill, Black & Lipsey, 2008). Our findings are consistent with those of a 
study of another approach to supporting ELs within the mainstream classroom, Project QuEST; 
researchers found no significant impact overall but did find an impact when ELs’ outcomes were 
examined separately (August, Branum-Martin, Cardenas-Hagen & Francis, 2009). 

Two limitations to this study deserve mention. First, it assesses impact when only fifth- 
grade teachers implement Project GLAD, when it is more likely that a whole school would 
implement at the same time. Secondly, retrospective power analyses with actual (rather than 
projected) ICC and R-squared values suggest that we were underpowered for some of our 
analyses. Data from a second year of implementation may help us learn more about the impact 
on student achievement. 
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Appendix B. Tables and Figures 


Table 1 

Project GLAD's 35 strategies in four component areas 


Focus & Motivation 
Strategies: Set 

standards for behavior, 
build engagement, 
interest, & prior 
knowledge 

Input 

Strategies: 

Provide 
information in 
multiple formats 

Guided Oral Practice 
Strategies: Scaffold 
student understanding, 
use of key vocabulary 
and language structures 

Reading & Writing Strategies: 

Scaffold student academic 
literacy 

Three behavior 

Graphic 

Chants 

Story map 

standards/super scientist 

organizer 

Poetry 

Team tasks 

awards 

Pictorial input 

Found poetry 

Process grid 

Cognitive content 

chart 

Sentence patterning 

Cooperative strip paragraph 

dictionary (CCD) 

Narrative input 

chart 

Writers’ workshop 

Observation charts 

chart 

Picture file card sort 

Group frame 

Teacher-made big 

10/2 lecture 

Exploration report 

ELD retell 

books 

Read 

Retells 

Clunkers and links 

Inquiry chart 

aloud/shared 
book experience 
Realia 

Team tasks 
Expert groups 
Mind maps 

Focused reading with personal 
CCD 

Ear-to-ear reading 
Listen and sketch 
Learning logs 
Interactive journals 
Home/school connections 


Table 2 

School characteristics for treatment and control groups 



Treatment (n=15) 

Control (n=15) 

Percent rural 

47% 

47% 

Percent FRL 

59% 

58% 

Percent white 

64% 

63% 

Percent Hispanic 

31% 

33% 

Average enrollment 

487 

464 


Source: U.S. Department of Education, National Center for Education Statistics. Common Core of Data: Public 
Elementary and Secondary School Universe, 201 0-1 1 , version 2a. 


Table 3 

Baseline comparison of teachers in the treatment and control groups 



Treatment (n=42) 

Control (n=50) 

Average years experience (SD) 

11.6 (8.4) 

17.2 (11.7) 

Percent female 

85% 

82% 

Percent white 

95% 

96% 

Percent with master's degree 

25% 

31% 

Percent with ESL endorsement 

5% 

5% 

Prior Sheltered Instruction Observation Protocol (SIOP) training 

66% 

70% 


Table 4 

Demographic characteristics of students in the analytic sample (n=2,253) 



N 

% 

Female 

1093 

49% 

FRL eligible 

1464 

65% 

Special education 

221 

10% 

White 

1403 

62% 

Hispanic 

753 

33% 

EL (current and reclassified) 

297 

13% 
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Table 5 

Outcome assessments for Project GLAD cluster randomized trial 


Subject 

Measure 

Pretest administered? 

Reading comprehension 

Gates-MacGinitie Reading Comprehension 
(4 th edition) 

Yes 

Vocabulary 

Gates-MacGinitie Vocabulary, (4 m edition) 

Yes 

Writing 

6+1 Traits writing assessment in response to a 
science prompt (each trait scored separately) 

No 

General science achievement 

Idaho State Achievement Test (ISAT) grade 5 
science assessment 

No 

Specific science content 

End-of-unit assessment from Scott Foresman 
earth science grade 5 textbook 

No 


Table 6 

Reading comprehension and vocabulary pretest extended scale scores for treatment and control groups 


Assessment 

Group 

N 

Mean 

Pretest 

SD 

Reading comprehension 

Treatment 

1060 

498.90 


35.34 


Control 

1025 

494.74 


35.77 

Vocabulary 

Treatment 

1064 

496.11 


34.03 


Control 

1214 

492.11 


33.66 


Table 7 

Two-level model results , Year 1: Gates-MacGinitie reading comprehension and vocabulary 




Readina comprehension 




Vocabulary 



Fixed 

Coefficient 

SE 

t(df) p 

ES* 

Coefficient 

SE 

t(df) 

P 

ES* 

Effects 










Treatment 

2.72 

1.96 

1.39(28) 0.177 

0.07 

2.03 

1.50 

1.35(28) 

0.187 

0.06 

Pretest 

0.81 

0.03 

31.28(2021) 0.000 


0.89 

0.01 

62.99(2038) 

0.000 


Intercept 

506.85 

1.33 

381.65(28) 0.000 


511.18 

0.93 

547.97(28) 

0.000 


Random Effects Variance 

SD ¥W) 

P 

Variance 

SD 

1 ¥ (df) 


P 

School 

18.59 

4.31 75.50(28) 

0.000 

11.21 

3.35 77.63(28; 

1 0.000 

Student 

719.92 

26.83 


425.11 

20.62 



* Effect size was calculated using Fledges' g. 







Table 8 










Two-level model results , 

Year 1: 

6-Traits writing 










Trait 1 : Ideas 



Trait 2: Orqanization 


Fixed 

Coefficient 

SE 

t(df) p 

ES* 

Coefficient 

SE 

t (df) 

P 

ES* 

Effects 










Treatment 

0.14 

0.74 

1.94(28) 0.062- 

0.22 

0.75 

0.06 

1.21(28) 

0.235 

0.14 

Pretest 

0.01 

0.00 

13.04(2015) 0.000 


0.01 

0.00 

12.95(2015) 

0.000 


Intercept 

3.97 

0.05 

82.00 0.00 


3.75 

0.04 

94.87(28) 

0.000 


Random Effects Variance 

SD y^Jdi) 

P 

Variance 

SD 

¥ (df) 


P 

School 

0.04 

0.20 261.28(28) 

0.000 

0.03 

0.16 

244.08(28) 

0.000 

Student 

0.32 

0.57 


0.23 

0.48 






Trait 3: Voice 



Trait 4: Word Choice 


Fixed 

Coefficient 

SE 

t(df) p 

ES* 

Coefficient 

SE 

t (df) 

P 

ES* 

Effects 










Treatment 

0.03 

0.06 

0.54(28) 0.590 

0.07 

0.06 

0.05 

1.37(28) 

0.181 

0.16 

Pretest 

0.00 

0.00 

8.97(2015) 0.000 


0.00 

0.00 

8.95(2015) 

0.000 


Intercept 

4.08 

0.03 

127.88(28) 0.000 


4.05 

0.03 

149.06(28) 

0.000 


Random Effects Variance 

SD ¥~W 

P 

Variance 

SD 

¥ (df) 


P 
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School 

0.02 

0.15 332.11(28) 

0.000 

0.02 

0.13 274.68(28) 

0.000 

Student 

0.14 

0.37 



0.14 


0.38 



Trait 5: Sentence Fluency 



Trait 6: Conventions 


Fixed 

Coefficient SE 

t(df) 

P 

ES* 

Co- 

SE 

t (df) p 

ES* 

Effects 





efficient 




Treatment 

0.06 0.05 

1.20(28) 

0.239 

0.11 

0.04 

0.03 

1.05(28) 0.301 

0.07 

Pretest 

0.01 0.00 

13.70(2015) 

0.000 


0.01 

0.00 

14.252(2015) 0.000 


Intercept 

3.84 0.03 

132.96(28) 

0.000 


4.02 

0.02 

169.68(28) 0.000 


Random Effects Variance 

SD 7 

(df) 

P 

Variance 

SD yf (df) 

P 

School 

0.01 

0.12 183.70(28) 

0.000 

0.01 


0.08 98.33(28) 

0.000 

Student 

0.17 

0.42 



0.17 


0.41 


~ p<.10 









* Effect size was calculated using Hedges' g. 








Table 9 

Two-level model results, Year 1: Science 

Idaho State Achievement Test Scott Foresman End-of-Unit Test 


Fixed 

Effects 

Coefficient SE 

t (df) 

P 

ES* 

Coefficient 

SE 

t (df) 

P 

ES* 

Treatment 

1.27 0.85 

1 .49(28) 

0.148 

0.13 

0.50 

0.30 

1.70(28) 

0.101 

0.24 

Pretest 

0.56 0.02 

33.78(2115) 

0.000 


0.19 

0.00 16.78(2014) 

0.000 


Intercept 

208.97 0.66 

314.43(28) 

0.000 


6.05 

0.13 

47.98(28) 

0.000 


Random Effects Variance 

“sd 7 

(df) 

P 

Variance 

SD 

1 

(df) 

P 

School 

5.12 

2.26 286.71(28) 

0.000 

0.65 

0.81 

354.47(28) 

0.000 

Student 

44.40 

6.66 



3.32 

1.82 





* Effect size was calculated using Hedges' g. 


Table 10 

Two-level model results for English learners, Year 1: Gates-MacGinitie reading comprehension and vocabulary 


Reading comprehension Vocabulary 


Fixed 

Effects 

Coefficient SE 

t (df) 

P 

ES* 

Coefficient 

SE t (df) 

P 

ES* 

Treatment 

6.87 4.03 

1.71(27) 

0.099- 

0.24 

5.72 

3.28 1.75(27) 

0.092- 

0.21 

Pretest 

0.66 0.06 

11.30(263) 

0.000 


0.62 

0.06 10.85(266) 

0.000 


Intercept 

474.26 2.65 

178.73(27) 

0.000 


473.95 

2.42 195.88(27) 

0.000 


Random Effects Variance 

SD 

7 (df) 

P 

Variance 

SD 777) 


P 

School 

55.44 

7.45 50.01(27) 

0.005 

25.96 

5.10 38.00(27) 

0.078 

Student 

514.90 

22.69 



505.31 

22.48 




~ p<.10 

* Effect size was calculated using Hedges' g. 
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Table 11 

Two-level model results for English learners , Year 1: 6-Traits writing 





Trait 1 : Ideas 



Trait 2: Oraanization 

Fixed 

Effects 

Coefficient 

SE 

t(df) p 

ES* 

Coefficient 

SE 

t(df) 

p ES* 

Treatment 

0.21 

0.10 

2.02 (27) 0.053- 

0.32 

0.15 

0.08 

1.78(27) 

0.086- 0.2 
7 

Pretest 

0.01 

0.00 

4.78 (256) 0.000 


0.01 

0.00 

5.04(256) 

/ 

0.000 

Intercept 

3.54 

0.06 

58.92 (27) 0.00 


3.41 

0.06 

59.73(27) 

0.000 

Random Effects Variance 

SD tf r (df) 

P 

Variance 

SD 

t (df) 

p 

School 

0.03 


0.17 47.13(27) 

0.010 

0.02 

0.14 

42.84(27) 

0.027 

Student 

0.34 


0.58 


0.26 

0.51 






Trait 3: Voice 



Trait 4: Word Choice 

Fixed 

Effects 

Coefficient 

SE 

t(df) p 

ES* 

Coefficient 

SE 

t (df) 

p ES* 

Treatment 

0.02 

0.06 

0.36(27) 0.723 

0.05 

0.10 

0.06 

1.64(27) 

0.112 0.22 

Pretest 

0.00 

0.00 

4.80(256) 0.000 


0.00 

0.00 

3.90(256) 

0.000 

Intercept 

3.94 

0.05 

86.50(27) 0.000 


3.84 

0.04 

92.29(27) 

0.000 

Random Effects Variance 

””SD 7]df) 

P 

Variance 

SD 

7 (df) 

P 

School 

0.01 


0.10 47.38(27) 

0.009 

0.01 

0.08 

34.25(27) 

0.159 

Student 

0.12 


0.35 


0.16 

0.40 





Trait 5: Sentence Fluency 



Trait 6: Conventions 


Fixed 

Effects 

Coefficient 

SE 

t(df) p 

ES* 

Coefficient 

SE 

t (df) 

p ES* 

Treatment 

0.03 

0.04 

0.61(27) 0.545 

0.05 

0.01 

0.07 

0.15(27) 

0.882 0.02 

Pretest 

0.01 

0.00 

6.30(256) 0.000 


0.01 

0.00 

8.59(256) 

0.000 

Intercept 

3.56 

0.02 

150.48(27) 0.000 


3.76 

0.03 

122.61(27) 

0.000 

Random Effects Variance 

SD y^Jdf) 

P 

Variance 

SD 

7 (df) 

P 

School 

0.00 


0.00 17.40(27) 

>.500 

0.00 

0.07 

33.38(27) 

0.185 

Student 

0.24 


0.49 


0.20 

0.44 



~ p<.10 









* Effect size was calculated using Hedges' g. 







Table 12 










Two-level model results for English learners , Year 1: Science 







Idaho State Achievement Test 


Scott Foresman End-of-Unit Test 


Fixed 

Effects 

Coefficient SE 

t (df) 

P 

ES* 

Coefficient 

SE 

t (df) 

P 

ES* 

Treatment 

0.88 0.85 

1 .04(27) 

0.309 

0.12 

0.42 

0.40 

1.05(27) 

0.303 

0.19 

Pretest 

0.44 0.04 

10.34(275) 

0.000 


0.02 

0.00 

3.34(266) 

0.001 


Intercept 

201.47 0.65 

309.24(27) 

0.000 


5.32 

0.25 

21.23(27) 

0.000 


Random Effects Variance 

SD yf 

(df) 

P 

Variance 

SD 

7 (df) 


P 

School 

1.98 

1.41 41.20(27) 

0.039 

0.65 

0.81 

74.69(27; 

) 0.000 

Student 

32.38 

5.69 



3.93 

1.98 





* Effect size was calculated using Hedges' g. 
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